Top 10 Popular Open Source Intelligence (OSINT) Tools Introduction
In this article, we will be discussing various OSINT tools that are available in the market. When we search the internet there are multiple pages of results that are presented. We just have a look at the first page and research and if we do not get what we are expecting, we stop right? But have you ever wondered what lies in those hundreds of pages of result? "Information" Let's get this free information using various tools. Tools are important but not knowing the usage of a tool will leave the user helpless. Before digging into the tools let's have a fair idea of what OSINT is and what can be achieved out of it.
What is Open Source Intelligence?
OSINT stands for open source intelligence. The Internet is an ocean of data which is an advantage as well as a disadvantage.
Pros are that the internet is free and accessible to everyone unless restricted by an organization or law. The Internet has all the information readily available for anyone to access. Cons are that the information is available that can be misused by someone with a malicious intent. Collection and correlation of information using these tools are referred to as open source intelligence. Information can be in various forms like audio, video, image, text, file etc. Below is the bird's eye view of the data categories available on the internet:
1.Social media websites like Twitter, Facebook etc. hold a lot of user data.
2.Public facing web servers: Websites that hold information about various users and organizations.
3.Newsletters and articles.
4.Code repositories: Software and code repositories like Codechef, Github hold a lot of information but we only see what we are searching for.
Why do we need tools?
Getting to know that the information is available is one thing. Collection of the information is second and making an analysis or intelligence out of them is the third. The information can be gathered manually as well but that will take the time that can instead be used in the later stages. Tools can help us gather the data from hundreds of sites in minutes and thus easing the collection phase. Let us say that the task is to identify whether a username is present and if so, on which all social media websites. One way is to log in to all the social media websites (I bet you don't know all of them!) and then testing the username in that. Another way is to use an open source tool that is connected to various websites more than what we can remember and checks the usernames presence on all the websites at once. This is done just in seconds. Run multiple tools to gather all target related information that can be correlated and used later.
OSINT Tools
1. Maltego
Maltego is developed by Paterva and is used by security professionals and forensic investigators for collecting and analyzing open source intelligence. It can easily collect Information from various sources and use various transforms to generate graphical results. The transforms are inbuilt and can also be customized based on the requirement. Maltego is written in Java and comes pre-packaged in Kali Linux. To use Maltego, user registration is required, the registration is free. Once registered users can use this tool to create the digital footprint of the target on the internet.
2. Shodan
Google is the search engine for all but shodan is the search engine for hackers. Instead of presenting the result like other search engines it will show the result that will make more sense to a security professional. As a certified information security professional one of the important entity is digital asset and network. Shodan provides you a lot of information about the assets that have been connected to the network. The devices can vary from computers, laptops, webcams, traffic signals, and various IOT devices. This can help security analysts to identify the target and test it for various vulnerabilities, default settings or passwords, available ports, banners, and services etc.
3. Google Dorks
Google is one of the most commonly used search engine when it comes to finding stuff on the internet. For a single search, the results can be of various hundred pages sorted in order of relevance. The results vary from ads, websites, social media posts, images etc. Google Dorks can help a user to target the search or index the results in a better and more efficient way. Let us say that the user wants to search for the word usernames but only requires the results with PDF files and not websites.
Some of the other indexing options are:
Inurl: search for a string in URL of the page.
Intitle: To search the title for a keyword.
Ext: To search for a particular extension.
Intext: Search for a particular text in a page.
Sometimes it is also referred to as Google hacking.
4. The Harvester
A harvester is an excellent tool for getting email and domain related information. This one is pre-bundled in Kali and can be very useful in fetching information. Below is an example of the output when we try to search for emails for Microsoft in PGP server. You can explore more as per requirement.
E.g the harvester –d Microsoft.com –b pgp
5. Metagoofil
Metagoofil is written by Christian Martorella and is a command line tool that is used to gather metadata of public documents. The tool is pre-bundled in Kali Linux and has a lot of features searching for the document type on the target, local download, extraction of metadata and reporting the results. For example: Users can scan for a particular kind of documents on a particular domain. Metagoofil –d nmap.org –t pdf.
6. Recon-ng
Recon-ng is a great tool for target information collection. This is also pre-bundled in Kali. The power of this tool lies in the modular approach. For those who have used Metasploit will know the power of modular tools. Different modules can be used on the target to extract information as per need. Just add the domains in the workspace and use the modules. For starters, here is a sample of the tool helping you.
7. Check Usernames
Social networking websites hold a lot of information but it will be really boring and time taking task if you need to check whether a particular username is present on any social media website. To get such information there is a website www.checkusernames.com. It will search for the presence of a particular username on more than 150 websites. The users can check for the presence of a target on a particular website so as to make the attack more targeted.
A more advanced version of the website is https://knowem.com which has a more wide database of more than 500 websites along with a few more services?
8. TinEye
Tineye is used to perform an image related search on the web. It has various products like tineye alert system, color search API, mobile engine etc. You can search if an image has been available online and where that image has appeared. Tineye uses neural networks, machine learning, and pattern recognition to get the results. It uses image matching, watermark identification, signature matching and various other parameters to match the image rather than keyword matching. The website offers API extensions and browser extensions as well. You can simply visit the image and right click on it to select search on tineye. Link: https://www.tineye.com
9. Searchcode
Searching for text is easy as compared to searching for a code snippet. Try searching for a code sample on google and you will be prompted with no results or irrelevant results. Search code offers you a feature to search for a line of code which could have been present in various code sharing websites like Github etc. Users can search for functions or methods, variables, operations, security flaws and anything that can constitute a code segment. Users can search for strings as simple as "a++" too complex methods. The search results can be further filtered basis a particular repository or language. Do consider a few things before you hit search.
10. Recorded Future
Recorded Future is an AI-based solution to trend prediction and big data analysis. It uses various AI algorithms and both structured and unstructured data to predict the future. The users can get past trends and future trends basis OSINT data.